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SECTION 1 


INTRODUCTION 

Filters sre devices that process incaning signals 
so as to eliminate noise, carry out smoothing, identify 
signals as belonging to different classes, or predict the 
next input signal based on present and past inputs. Filters 
may be divided into two classes; fixed and adaptive. The 
impulse responses of fixed filters do not vary with time, 
whereas that of adaptive filters are, in general, time- 


Ihe design of fixed filters is based upon prior 
knowledge about signals and noise. Adaptive filters, on the 
other hand, have the ability to adjust their own parameters, 
and their design does not require prior knowledge of signal 
and noise characteristics. These are, in essence, a class of 
"learning machines" whose parameters are self “ad jus table in 
accordance with estimated or measured statistical character- 
istics c£ input and output signals. 

The classical theories of detection and estimation 
are usually presented in a fixed or non-adaptive context. In 
such a context it is assumed that seme statistical properties 
of signal and noise are known beforehand. In the Wiener 
theory of estimation, for instance, one needs to know the 
second-order properties, viz. the correlation functions of 
signals. Similarly, in detection theory, the optimal receiver 
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makes use of a likelihood ratio which requires signal and 
noise probability distributions. 

In practice a priori knowledge of signal and noise 

characteristics may not readily be available. Moreover, if 

the signal and noise parameters change, as is usually the case, 

the original fixed optimum receiver will no more be optimum, 

« 

resulting in degradation c£ performance. Adaptive filters 
have found extensive use in such applicaticns These 

filters "learn" the characteristics of the signals initially 
and track them, if they vary slowly enough, by adjusting their 
impulse responses. Adjustment is accomplished through an 
algorithm that responds to a suitably derived error signal. 
Thus, with a proper algorithm the filter can operate under 
changing conditions and can readjust itself continually to 
minimize the error signal. Adaptive filters have been used in 
adaptive smoothing, filtering, equalization, prediction, and 
estimation of signals, and identification of system parame- 
ters; they have also been used in spectral estimation, in noise 
whitening and in speech signal processing Ll"2, l-Sj, 

A particular application of adaptive filters is in 
"noise cancelling". It is an alternative method of estimating 
a signal corrupted by additive noise. A "reference input" is 
derived from one or more sensors located in the noise field 
at points where the signal is weak or undetectable. The 
reference input is adaptively filtered and subtracted from a 
"primary input" consisting of the signal and the additive 



noise. This results in an output that is the best estimate, 
in the mean--square sense, c£ the actual or desired signal. 
Adaptive filtering before subtraction alla-;s processing of 
signals that are deterministic or stochastic, stationary or 
nonstationary , 

This thesis deals with the theory c£ adaptive noise 
cancellers, their design, and their implementation by means 
of microprocessors. In Section 2, tlie principles of noise 
cancelling are reviev/ed and the various design rules avail- 
alole at present for adaptation based on the so-called LMS 
algorithms are brought xi/ithin cne framework. Then the 
relative merits of some recent versicsos of the LMS algorithm 
are discussed, and new computationally efficient algorithms 
are proposed. 

After discussing the advantages and limitations in 
Section 3, the noise cancelling technique is illustrated in 
Section 4 by considering various applications and presenting 
simulation results. In Section 5, t±ie possibility of using 
microprocessors for implementing noise cancellers is examined 
The hardware and software aspects of implementing a typical 
noise canceller are discussed; also, a real-time noise 
canceller scheme is proposed. The thesis concludes in 
Section 6 by summarizing the results and with suggestions 
for further work. 
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SECraON 2 

PRINCIPLES and ALGORIIHMS 

2.1 Ilie Concept 

Ihe concept of adaptive noise cancelling may be 
explained with the aid of Figure 2.1, A signal s, transmi- 
tted over a channel and corrupted by an additive noise or 
interference n^, forms the primary input to the adaptive 
noise canceller (ANC), A noise n^/ derived from the same 
source field as the signal, forms the reference input, the 
sensor for n being so placed as to pick up only the noise 
components. The reference noise is uncorrelated with the 
signal, but correlated in some unknown way with the primary 
noise, 

Ihe noise n„ is filtered by means of an adaptive 
filter to produce an output y that is a close replica of n^. 
Ihe output of the filter is then subtracted from the primary 
input to produce the system output, z. The objective is to 
produce a system output that is the best fit, in the mean- 
square sense, to the signal s. In the figure, the feedback 
from the system output to the adaptive filter is meant to 
provide requisite parameter adjustments. It is assumed 
that s, n^, n^ and y are statistically stationary and have 


zero means 
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Referring to Figure 2,1^ the system output is 
2 = s + - y . 

Ihe mean-square value of the output is 

e[ 2 ^] = E[s^] + E[(n^-y)^] + 2E[s(n^-y)] , 

If n is uncorrelated with s, then y and hence n,-y will 
also be uncorirelated with s. Accordingly, 

E[z^] = E[s^] + El{n^-y)^] . (2.1) 

Ihe mean— square error (mse), e, between the output and the 
desired signal is 

E[e^] = E[(z-s)^] 

= E[(n^“y)^] . (2.2) 

Inasmuch as the signal is unaffected by the adaptive filter, 
it is seen fran Eqs, (2.1) and (2,2) that minimizing the rase 
is equivalent to minimizing the mean-square value of the 
system output. When e[z^} is minimum, E[(n^-y) j = 0, 

Iheref ore, y s= n^^ and z = s. Hence, minimizing the mean- 
square value of the output causes the signal to b® perfectly 
noise free. 

When the reference input is completely uncorrelated 


with the primary input. 
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E[z^] = E[(s + n^)^] + 2E[- y(s + n^)] + E[y^] 

= e[(s + n^)^] + E[y^] . 

In this case, the mse is minimized if E[y ] is minimized, 
which is acGOT^jlished by the filter by making its output 
zero* 

Ihe. adaptive filter, indeed, so adjusts its 
parameters as to have, ideally, the inverse of the channel 
characteristics, thus reproducing the correlated noise 
coDpcnents that originated at the source field and were 
degraded by the channel. It may be noted that, though we 
have c CXI side red cxily the case of stochastic signals, the 
discussion is equally valid for the case of deterministic 
signals, as will be seen in Secticxi 4, 

It is now e 3 q)lained hc3W the minimizaticxi of the n'.ean“ 
s<juare cutput(i,e,, mse) is actually carried out. For the 
sake of discussion, the signals are assumed ergodic, 

2,2 Ihe Wiener Soluticxi 

Let the jth sairple of the primary input be denoted 
by d(j), Ihe jth set of reference samples may be repre- 
sented by a randcxn vector where 

X ^ J ^ ~ 3 ^ ^j)/ •••/ ^ 3 ^ 3 • 

A noise estimate, y(j), may be obtained as a linear conbi- 
naticxi of the reference samples, X(j); i.e,. 
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y(j) = AU) = W 

where the canponents c£ the vector 
W = [ w^, ] 

are the weights whose values have to be determined in accor- 
dance with a specific criterion c£ performance. If only the 
seccnd=order properties of the signals are known, the 
sinplest possible criterion is the minimization c£ mse, 
which is accctnplished by minimizing the output power. The 
corresponding weight vector W is then determined with the 
help of the Wiener filter theory. 

The output at the jth instant is given by 

z( j) = d( j) • (2.3) 

Squaring Eq. (2,3) and taking expected value cf both sides 
yield 

E[z^(j)] = E[d^(j)] - 2E[d(j) x'^(j)] W 

x'^( j)] W . (2.4) 

(2.5) 




+ wXx(j) 


Let 




t E[d( j) X( j) j 

d(j) X^(j) 

<3(j) x^(j) 

= E ‘ ^ 
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where R, denotes the vector of cross-correlations between 
— dx 

the reference samples a^nd the primary input/ and 


x^(j) 

3<i(j) XQ(j) x|(j) 


( 2 . 6 ) 


X 


n-1 


(j) 


where R_ is the covariance matrix o£ the reference samples. 
Now, the mean-square output may be expressed as 

E[z^(j)j = E[d^( j)] - 2R^^ W + W . (2.7) 

Follov/ing the usual procedure/ the condition for minimum 
output power is found by differentiating Eq. (2.7) with 
respect to the weight vector and equating it to zero. 
Accordingly/ the gradient is given by 

7E[z^(j)] = - 2^^ + 2R^ W 


where 
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The optimal weight vector, W*, is obtained by equating the 
gradient to zero. Therefore, 


The matrix R , which is Toeplitz, is generally positive 



definite and, therefore, invertible. Accordingly, the above 
relation may be written as 


W* 





which is the Wiener-Hopf equation in matrix form. 

In practice a direct solution for the weight vector 
is difficult to obtain, for an nxn matrix needs to be 
inverted and n(n + l)/2 autocorrelation and cross-* 
correlation measurements have to be made to obtain the 
elements of and R^^. This evidently would entail a huge 
computational burden as well as storage of large amounts 
of data. Furthermore, this procedure needs to be repeated 
if the signal statistics change. A number of algorithms 
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have therefore been prqposed which ccnpute the Wiener 
solution iteratively based on the incoming data samples. 
Among them is the so-called LMS algorithm, which is simple 
fron an implementation point of view, yet effective in many 
noise cancelling problems [2-1], This algorithm is chosen 
for discussion here both to gain an insight into the 
behaviour a£ AWC's and to obtain simplified design rules. 
Sane other, recent algorithms, which are of practical 
interest, are also discussed and canpared with the LMS 
algorithm towards the end of this section. 

2.3 Ihe LMS Algorithm 

Equation (2.7) indicates that, for stationary input 
signals, the mse is a second-order functicn of the weight 
vector and, therefore, is a convex function, the minimum 
of which will correspond to the cptimum weight values. 

To find this minimum/one of the gradient methods, 
namely the steepest descend method, is usually employed, 

Ihe method is perhaps best explained by considering a 
one-dimensional case shown in Figure 2.2. Suppose that 
w(j), the weight value at the jth instant, has some value 
different from w* . We find the slope oE the curve at this 
point. If the slope is negative, w(j) is to the left of 
the minimum, and if it- is positive, w(j) is to the right. 
Therefore, if at the next time instant w(j) is chosen as 



1 


Fig. 2 .2: Variation of msc with rth filter weight- 
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,/ • 1 \ / • \ dE [ 2^1 

w(j +1) = w(j) - y dw""^ — 

I W = W ( j ) 

Where y is a constant, then w( j + 1) is shifted in the 
right direction to reduce the mse. 

For the n-dimensiaial case, where the gradient 
V E [z (j)] rather than the slope is a£ interest, the updating 
algorithm becomes 

W(j + 1) = “ V (2.9) 

Equation (2,9) implies simultaneous adjustment of all the 
filter weights. Ihe adjustment of each of the filter weights 
is prgoortional to the corresponding conponent of the gradient 
vector. Interchanging the operations of expectaticxi and 
differentiation yields 

w(j + 1) = w(j) - 2y VE[2(j) V 2 (j)]. 

Fran Eq, (2.3), we obtain 

Vz( j) = - X( j) 

Therefore, 

W(j + 1) = W(j) 4 2y E[z(j) X(j)] , (2.10) 

Which shows that the incremental weight update is a cross- 
corirelati on between the error sequence and the input data 
sequence, when the sequences are ergodic. Ibus the 
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process of cptimization can be pictured as the process of 
reaching the point in time when the error sequence is 
orthogonal to the data sequence. 

In implementing Eq. (2.10), the evaluation of the 
cross“Correlation terms requires a large amount of corpu- 
tation as well as storage. A particular form of the above 
algorithm that overcomes this difficulty has been introduced 
by Midrow and Hcpf [2-1]. The algorithm makes the approxi” 
mation that the gradient of a single time sample of the 
squared output is an estimate of the gradient of the mean 
square function; i.e,, 

'^eEz^Cj)] (V . (2.11) 

Such an estimate is unbiased; i.e,, 

E { V[z^( j)] } = VE[z^( j)] . 

Equation (2.9) now becomes 

W(j + 1) = W( j) + 2 vz( j) X( J) (2.12) 

where 

z( j) = d(j) - w'^( j ) X(j) . 

The above algorithm means that the next weight 
vector is obtained by adding to the present weight vector 
the input vector scaled by the present value of the output. 

It is simple and easy to implement. Although it makes use cxf 
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gradients of rase functions, it does not require squaring, 
averaging or differentiation. It is commonly referred to 
as the LMS algorithm. Ihe constant p is known as the 
adaptation constan t or step size. Ihe realization of 
the above algorithm by means of an adaptive filter is shown 
in Figure 2.3. The components of X(j) are obtained as a 
delayed sequence of the reference s;^ples. 

By analyzing the LMS algorithm, expressions for rate 
of adaptation and deviation from the least rase after convert 
gence can be obtained. Ihe analysis may be done either in 
the sample domain or in the frequency domain. We adopt the 
sample domain approac±i. 

2,3,1 Rate of Adaptation and Learning Curve 

Taking expected value of both sides c£ Eq. (2.12) 

yields 

E[w(j + 1)] = EfwC j)] + 2 pE[dXj) X( j) ] 

2P E[x(j) x'^(j) w(j)] . (2.13) 

It is assumed that the time between successive iterations 
is sufficiently long so that X(j) uncorr- 

elated. The weight vector W(3) is then uncorrelated with 
X(j), Equation (2.13), therefore, takes the form 

ECw(j + D] = E[w(j)] + E[w(j)] 




ALGORITHM 
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i.e. 


= [I - 2y R^3 e[w( j)] + 2y . 

With an initial vector W(o)/ the above expression may be 
iterated j times to obtain 


j + 1) ] = 


(I - 2 V R ) W( o) 

!X 


,m 


+ £ (I -2uR^) 

m=o 


(2.14) 


Conventional analysis transforms Eq. (2.14) to an 
orthogonal coordinate system where the covariance matrix R 
is diagonalized; the result is that each weight can be 
expressed independent of the other weights. Expressing R 
as 

R^ = A Q = q'^’a Q (2.15) 


where A is the diagcnal matrix of eigenvalues, and Q, the 
orthonormal matrix of eigenvectors, we obtain fron Eqs. 
(2.14) and (2.15) after simplification, 

QE[w(j + 1)] = (I - 2yA )^+^ Q W(o) 


j 

+ 2u Z (I 
re=o 


2y A )"^ Q 


— dx 


(2.16 ) 


Let j -»■ «> . Ccnsider now the term 

1 (I - 2PA)”^ . 

m=o 


(2.17) 
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Since each diagonal element of the matrix 

(I " 2 yA) forms a geanetric series, the ith element, X^, 

may be expressed, using the standard summation foimiula, as 


I (1 


2 V 


in=o 




2y 


X. 

1 


provided that 

1 1 *- 2V X^ I < 1 . 

Similarly, 

00 

I (I - 2yAf = A“^ 

ni=o 

so long as 


(2.18) 


1 


2 y X, 


max 


< 1 


O < y < (2.19) 

^max 

where X is the maximum eigenvalue of R ,. Similarly, if 
max X 

the inequality (2.19) is satisfied, the RHs first term of 
Eq. (2:, 16^ equals zero as j . Therefore, 

lim E [W(j + D] = R^x ' 

Thus, as the number of iterations increases, the 
mean weight vector converges to the Wiener solution. The 
equatiai may be interpreted as follows: If we consider an 
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ensemble of adaptive processes, each having individual 
stationary ergcdic inputs drawn frcm the same ergcdic 
process, with all initial weight vectors equal, the weight 
vector, obtained by averaging over the ensemble at iter- 
ation j, will converge to the Wiener solution, in the limit. 
In practice it is difficult to know the value of 
to set the bound on y , However, a sufficient coidition 

lUCiX 

for convergence may be obtained by noting that 


X < 

max — 


R 


X 


where t R , the trace of R , corresponds to the reference 
input power, which is generally known. Therefore, a 
sufficient condition to ensure convergence of the weight 
vector is 


O < y < 


t„ R. 


( 2 , 20 ) 


The time constant, of the ith weight is defined 
as the time taken for the weight to get to l-e”^ of its 
steady-state value for zero initial conditions and step 
inputs. Here, the unit of time is the time taken for one 
iteration cycle. For the ith weight, its value at the jth 
instant is the sum of terms of the geometric series 
(2,16) with geometric ratio 1-2 y X^, Therefore, 

1 - (1 - 2y X^)^ { _ = 1 - e“^ 
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or, equivalently. 


T . 
i 


in -(I - 2 y x^) 


In most practical situations !2ii X^l << 1, in which case. 


ln(l-2yX.) _2uX. 

1 — 1 


.e,. 


1 

"2 ii "X , 


i=h,l, ... n-»l , 


( 2 . 21 ) 


Hence the transients consist of a linear ccmbination of 
exponentials with time constants given by Eq. (2.21), 

The progress of an adaptive process can be moni- 
tored by displaying its "learning curve ”. It is a. plot of 
the expected value of -the mse at each stage of the learning 
process as a function of the number of adaptation cycles. 
The underlying transient phenomenon that takes place in 
the weight values is exponential. Iherefore the mse, which 
is a quadratic function of the weights, has also an expo- 
nential decay. The time constant of the learning curve is 
determined as follows: 

Combining Eqs. (2.7) and (2.8) yields 


E [z (j)] 


min 


= E [d^(j)] 




■Sdx 


( 2 . 22 ) 


From Eqs. (2.7), (2.8) and (2.22), we obtain 

E[z^(j)] = E[z2(j)]^^^ + (W(j) - R^(W(j) - W*) 


(2.23) 
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It is seen that the mse is quadratic in the weight vector 
and, hence, has a geanetric ratio, (1 - 2W A ) . 0116 time 

constants of the learning curve are, the ref ore. 


T - 

^mse 


'\j 


1 

iirq 


1 ” ^ ^ • n^l^ 


(2.24) 


If the eigenvalues do not differ widely, it is reascnable 
to express the time constants c£ the learning curve as a 
function of the average eigenvalue, where 


av 


^ n-1 

" n ^ ^i 
i=jO ^ 


t R 
r X 

n 


Accordingly, Eq. (2.24) takes the form 

%se ^ T v - T; - <2.21 

Which gives a formula for the average or effective time 
constant of the learning curve that is useful even when 
the eigenvalues are highly disparate. This time constant 
is referred to as the " rate of adaptation " in the literature 
(A more appropriate term might be adaptation time). 

When actual experimental learning curves ere 
plotted, they are generally of the form of noisy exponen- 
tials because of the noise due to statistical averages 
taken with limited sample sizes. This point is discussed 


next 
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2.3.2 Ma ladjus tm ent with LMS Adaptaticai 

It is o£ interest to know the penalty incurred in 

making the approximation of (2.11), T'iJheroas the mean- 

weight vector^ under appropriate conditions, converges to 

its proper value in the limit, its variance often induces 

a certain amount of degradation. Ihis degradation in 

perf orraance is measured in terms o£ a dimensionless quantity, 

kno^^/n as " misad justment ” [2-2] . It is defined as the ratio 

of the average excess mse to the minimum mse. An expression 

for the misad justraent may be obtained under the steady- 

state conditions as follows: 

As before, it is assumed that the successive data 

vectors X‘s are uncorrelated over time, and that 

2 

W(j) 0 . W** The true gradient VE[z (j)] may be expressed 

^ 2 

as a sura of the gradient estimate v[z (j)] and a zero- 
mean gradient estimate noise vector/N(j); i.e., 

= ^[2^(j)] +N(j) (2.26) 

2 n ^ 

where ?(j) ^ Efz (j)]. Substituting for V yields 
N( j) = - 2z(j) X( j) + ( j) - 

Also. v7hen W(j) = jf*. Vg(j) =0. Iherefore. 

M( j) = - 2z( j) X(j) . (2.27) 

Ihe covariance matrix of N(j) is 
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i E[N(j) N'^{j)] 

= j) X( j) j)] . 

In accordance with the orthogonality theorem, when W(j) = 
2(j) and X(j) uncorrelat«3 . if they are assumed zero- 

mean Gaussian, then z(j) and XO^ statistically indep- 

endent. Therefore, 

R^(j) = 4E[z2(j)]R^ 

or, in the steady-state. 


= 45. R . 

min X 


(2.28) 


Since X*s are stationary and uncorrelated over 
time, it follows that the gradient noise vector N(j) is 
also stationary and uncorrelated over time. Equation (2.28) 
may be rewritten as 


% 


4 5 . A 
min 


(2.29) 


where 


N* ( j) = Q N( j) . 

Such a transformation makes the analysis simpler. 

The effect of gradient noise aa the steady-state 
wei^t vector can now be determined. The LMS algorithm 
with a noisy gradient estimate can be expressed as 
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W( j + 1) = w( j) y V[z2(j)] 

= W(j) “ V [Vc(j) - N(J)] . (2.30) 

Equaticn (2,23) may be reexpressed as 

= 'min J> «x2<J) <2.31) 

where 

yCJ) i ifU) - W*. (2,32) 

An alternative expression for the true gradient , obtained 
fron Eq. (2.31), is 

VE(j) =2R^V(j) . (2.33) 

From Eqs. (2.30), (2.32) ard (2,33), we obtain 
y(j + 1) = y(j) -y [(2i^y(j) -rN(j)] 

or, letting V*( j) = Q y( j ) , 

y'(j + 1 ) = y'(j) - 2yA y‘(j) + V N‘(j) 

= (I - 2yA ) y*(j) + yN*(j) . (2.34) 

Since N(j) is uncorrelated over time, N’(j) and y*(j) are 
also uncorrelated . Postmultiplying both sides of Eq, (2.34) 
by their transposes, and taking expected value of both sides 
yield 
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%(j + 1) A E[v'(j 1) V''^(j + D] 


= (I - 2 vA) R^{j)(I - 2uA ) 


2 o. 

y R ' 
n 


(2.35) 


The covariance matrix R^(j) is diagonal. If y*(j) 
is assumed to be stationary, Eq. (2.35) may be rewritten as 

r; = (I - 2y A )2 ^ 

Substituting for R^ from Eq. (2.29) and rearranging the 
terms yield 


(I -U A ) R> 
V 


V S . 
mxn 


I . 


Assuming that u << 1, which is consistent with the assump- 
tion that W( j) it is seen that 


u A < < I . 


The ref ore. 


R' 

V 


-v u E . I 
— ^ ^ min 


i.e. / 


R 


V 


= w 5 . I 

min 


(2.36) 


The above expression shows that, under the stated assump- 
tions, the ccmponents of the weight- vector noise are of 
equal variance and are mutually uncorrelated. 



I 
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Due to the randan noise, caised by the 
gradient estimate noise, the weight vector is on the 
average misad justed from its optimal setting. This results 
in an excess mse. From Eq, (2.31), an expression for the 
excess mse may be obtained as 

?exs “ A Z'(3) . 

The average excess mse is 

E[e ] = i. E[{v '(j)}h . (2.37) 

i=o 

From Eqs. (2.36) and (2.37) VJe obtain 

n-1 

E re 1 = y e . z x, 

^ exs min . ^ i 


= y 5 


min 


R 


X 


Hence the expression for the misad justment is, by defini- 
ti on , 


M 


E r c 


exs 


’min 


] 


i.e. 


= < 2 . 33 ) 

ymert the eigenvalues are sufficiently similar, 

an apprcximate expression for the misad justment in terms 

of T , obtained fron Eq. (2.25), is 
mse 
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M 


n 
4 T 

mse 


(2.39) 


Equations (2.2 5) and (2.39) provide approximate 
expressions for the rate of adaptation and the deviaticn 
from the minimum mse, respectively. Now the questicn arises 
as to how to determine the number of taps. The delay between 
the taps must be chosen in accordance v;ith the sampling 
theorem. Let B Hz be the bandwidth c£ the reference signal. 
The delay T is chosen as 



If Ab is the frequency resolution of the adaptive filter, 
then the ■ total real-time length of the delay line is 



Thus, 



(2,40) 


The expressi:ais (2.20), (2.25), (2.39) and 

(2,40) provide a set of approxiraate formulae for designing 
ANC's with adaptation based cn the LMS algorithm. The 
first three equations depend on the reference input power, 
which could be measured approximately by a time average 
pq^r of the reference samples. In practice same trial 
and error procedure is necessary to achieve desired results. 



A brief discussion about the overall performance of 
the ANC is now in order. In broad-band signal processing 
increasing the nurober of taps may improve the performance, 
for it permits the adjustment of gain and phase at many 
frequencies simultaneously. Nevertheless, for a fixed rate 
of adaptation a larger numbers of taps increase misad justment 
as is evident from Eq, (2.39), Similarly, increasing the 
adaptation constant speeds up the adaptive process, but it 
increases the misad justment, too. Alternatively, for a 
fixed n, more the reference input power the faster is the 
adaptation as seen from Eq. (2,25). Ihere is a tradeoff 
possible between the misad justment and the rate of adaptation. 

It has been pointed out [2“2] that the rate of adaptation is 
optimized when the excess mse resulting from adapting too 
rapidly equals twice the excess mse resulting from adapting 
too slowly. When a slow, but precise adaptation is desired 
is chosen such that 

0 < p, r • (2,41) 

Although the assumption of independent successive 
samples is invoked in the convergence proof for the mean 
weight vector, Daniel [2-3] has shown that adaptation using 
highly correlated successive samples does converge to the 
Wiener solution. However, it leads to slightly higher 
steady-state mse than does adaptation using statistically 
indeperdent successive samples. Also, Shensa [2-4] has 
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pointed out that the Wiener solution is a good approximation 
for the steady— state weight vector as long as it is larger 
than the misad justment; but should it be zero, one may not, 
in general, ignore the misad justment no matter how small. 

In a stationary environment, it is possible to decrease p 
jfl-1] as the adaptation progresses bringing the gradient 
eventually to zero and so also the misad justment, 

There are indeed a host of other algorithms which 
might in practice prove useful. Some of the recent algorithms 
are discussed and compared in what follows, 

2 *4 The Modified LMS Algorithm (MLMS ) 

The LMS algorithm is conditionally stable with the 
result that the rate of adaptation and, in general, the 
performance of the filter are constrained, A modification 
[2-5] has been introduced in the LMS algorithm that removes 
the constraint on p. and still achieves the sane mean steady- 
state weight as the LMS algorithm. 

The recursive relation {2,12) is modified as 

W(j) = W ( j-»l) + 2p z(j) X( j) (2.42) 

z(j) = d(j) - w'^(j) X(j) . 

That is, the present weight is obtained from the 

present values of 2:(j)' and x{j) in contrast to the LMS 
algorithm which generates the -.next weight from the present 
values of z(j) and X(j), The condition for stability of the 
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one-dimensional case of idie algorithm is given by [2—5] 

1 . - 
O 1 

1 + 2ji |xl 

which is always satisfied if ^ and |x j are non— zero* 

Although the experimental performance of this algo- 
rithm has been reported to be better than the LMS algorithm^ 
its implementation is limited by the ccxnputation involved; 
the authors [2-5] have suggested that one way is to compute 
each weight in a closed-loop fashion vhile the other weights 
are frozen; i.e,, 

w^{j) = w^(j-l) + 2# zCj) x(j-i) 

n 

z(j) = d(j) -w^(j) x(j-i) - S ^^3) - 

m=l f m^i 

Ihe actual output signal resulting from this procedure is 
then taken to be 

z(j) = d(j) - E w.(j) x{3-i) . 

i=l ^ 

An alternative and perhaps more elegant way of doing 
this is to invoke the standard matrix inversion formula that 
results in 

[l + 2\x X(j) x‘^(3)]“^ = I - ex(3) x'^Cj) ^ 

where c> 1 + X^(.5) 2(o) 


(2.43a) 
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is a constant, and compute Eq, (2,42) according to 


A(j) 

6 I “ C X( j ) X*^( j ) 

(2,43b) 

W( j) 

= A(j) [W(j“l) + 2lid(j) X(j)] 

(2.43c) 

z(j ) 

= d(j) - w’^( j) X{ j) , 

{2.43d) 


But this still leaves considerable computation to be performed 
and consequently makes real-time signal processing difficult 
to achieve, A further simplification in computation is 
described later on , 

In an implementationbis^ on digital processors it 
is mainly the number of multiplications in terms of which 
the computational complexity is assessed. With a view to 
reducing the number of multiplications in the case of LMS 
algorithm, various alternative algorithms have been proposed 
in recent years, Ihese mainly fall into two categories: 
what may be called, the “clipped -information” algorithms, 
and frequency domain algorithms, These are discussed next, 

2,5 The Clipped-data LMS Algorithm 

The clipped-information algorithms make use of 
infinitely quantized data or output in the weight update 
equation (2-12) thereby eliminating the need for multi- 
plication, When the data is quantized or "clipped," the 
following algorithm, known as the clipped-data LMS (CLMS) 
algorithm [2-6], results: 
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y(j) = 

n-1 

1 w.(J) X (j-i) 

i=0 

{2.44a) 

z(l) = 

<a(j) - y(J) 

(2.44b) 


= w^(j) + ^ z(j) Sgn x(j-i) . 

(2,44c) 


This elimination o£ n multiplications in the above 
relations provides a distinct advantage in terms of speed of 
operation of the processor without any sacrifice in the 
level of convergence. However, the speed of adaptation is 
stated [2-6] to have been decreased in comparison with the 
LMS algorithm by a factor of */2; it has also been mentioned 
that the steady-state values of the weights arK3 the time 
constant depend on the amplitude levels of the reference 
signals rather than their power levels. 

For the purpose of ccmparison with the other algo- 
rithms, the properties of the CLMS algorithm were derived. 

The derivation is included in Appendix A, It is seen that 
the derivation leads to a time constant different from that 
mentioned in [2-6], The algorithm was simulated to conf irm 
the results presented here. It was verified that for Gaussian 
inputs . the actual time constant is given by the relation 
derived in the appendix. 

Without loss of generality, the properties of the 
one«K3imensional case of the LMS, MLMS and CLMS algorithms 
may be compared. These are given in Table 2,1, 
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Table 2*1 : Ccanparison of properties of algorithms. 


Algorithm 


Steady-state 

weight 


Condition for 
convergence 


Time constant 


LMS 

«<lx 

O < 11 < - 

1 


U < n < ~ 



MLMS 


Always converges 

1 




CLMS 

R R , 

X dx 

o < u < ^ ^ 

Ox 


^ ^ ^ 72 




It is seen that all the three algorithms converge to 
the Wiener solution in the limit/ under the stated assumptions. 
The time constants of the LMS algoriibm and the MLMS algo- 
rithm are the same and that of the CLMS algorithm differs 
from the rest by a factor of ^ , whereas is the standard 
deviation of the reference input. In so far as the stability 
is concerned/ the MLMS algorithm proves superior to the other 
two. 

If/ instead of the reference data/ clipped output 
is used in the weight update equation (2,12)/ then/ what is 
known as the hybrid (HYB) algoritlm [2-6] results, 

Ihe computational saving in the ClidS algo ri tiros 
has been achieved at the cost of speed of convergence, vhich 
may be disadvantage's in certain applications. The 
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frequency dcanain a-lgorithins seek to overcone this difficulty 
for large values of n. Ihese are describei below - 

2.6 Frequency Domain Algorithms 

Ihe frequency domain algorititms evaluate Eq. (2,12) 
using FPT techniques. Two algorithms have so far been 
proposed, one [2-7] using circular convolution and the other 
[2-8] using linear convolution. The difficulty with the 
former is that, in general, it does not ensure convergence 
of the weight vector to the Wiener solution; whereas the 
latter does ensure. The latter algorithm, known as the facst 
LMS (FLi'iS) algorithm, is described here. 

Grouping the data in n-point blocks, the relation 
(2,12) may be iterated n times to obtain 


y(mn+i) = w”^(m) X(mn+i) (2,45) 

z(mn+i) = d(mn+i) - y(mn+i) (2.46) 

n-1 

W(m+1) = W(m) + 2\i I z(mn+i) X(mn+i) (2,47) 

i=o 


where m is the block index, n the radix, and i ^n-l. It 
is assumed that the change in the weight vector over one 
data block is negligible so that the weights need be updated 
only at the end of each data block, leaving them fixed within 
a block. 


The FLMS algorithm implsnents the above relations in 
the frequency domain. In as much as it is an exact implement 

_ _ ri; 'it'4; _ . 
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properties are the same as the LMS algorithm implemented in 
the time domain* Ihe FLMS algorithm [2—8} is summarized below. 
The convolution in Eq, (2,45) and the correlation in 
E*?* (2,47) need be carried out using 2n-jx)int FFT, letting 
k denote tiie discrete frequency variable, 

Pf^(k) £ FFT [w(m), O] 

T 

X (k) £ IFT [x(mn-n), ,,,, x(mn), x(mn+l), x(mn+n-l)] 

T 

Y (in) £ [y(mn), y{mn+n-l)] 

1 = Last n terms of FFT”" [w(k) 2£(k)] ' (2,48a) 

T 

2 (ra) £ [z(mn), z(mn+n-l)] 

ss [d(mn) “ y(ran), *,,/ d{im+n?-l) - y(mn4-n-l)] (2,48b) 

2 (k) A FFT [0, z(m)] 

l|(jR+>l) = W(m) + 2p. P (m) (2,48c) 

P(m:) £ First n terms of FFT [E(k) 3^ X(k)], 

where denotes the element-by-element multiplication of the 

two vectors. 


2,7. An HR LMS Algorithm 

The adaptive filter is usually implemented by means 
of a programmable non -recursive (FIR) filter. Such a filter 
can produce only zeros with no E»les in the filter transfer 
function. In applications where high selectivity is imper- 
ative, a relatively high-order transfer function has to be 
used to approximate the poles to a reasonable degree. For 
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the same filter specifications the required order in a non- 
recursive design can be as high as 5 to lO times that in the 
recursive design [ 2 - 9 ]. Alternatively, a high-order FIR 
filter may equivalently be realized by a low-order IIR filter, 
Ihis will result in considerable saving in ccmputation as well 
as cost of implementation, Feintuch [2-10] has proposed an 
adaptive IIR LMS (ILMS) filter that will prove useful in 
certain noise cancelling applications. Ihe ILMS algorithm is 
furnished by 


y(j) = 

x{j) + w^Cj) y(j) 

(2.49a) 

2 ( j ) ss 

<a(j)~y(j) 

(2.49b) 

( j+1 ) 

= + 2\i^ z(j) X(j) 

(2.49c) 


= n^ii) + 2^2 2 ( 3 ) Y(j) 

(2.49d) 


where 

x'^(j) s [x(j), 

1*^(3) = [y(J“i)/ ...r y(j-n 2 )] 

The block diagram of the resulting noise canceller is shown 
in Figure 2.4. 

An examination of the algorithms discussed so far 
suggests new possibilities of computationally efficient algo- 
rithms, Ihese are now outlined. 

2,8 New AlgoritlTOs 

Speed of a digital ANC is of concern when signals of 
large band widths have to be processed. For applications in 
which slow adaptation is acceptable, the CLMS algorithm has 
an edge over the others in terms of speed of processing, A 
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furtlier improvement in speed can be obtained by implementing 
the FLMS algorithm partly in the sample domain and partly in 
the frequency domain. This strategy results in an algorithm 
that possesses significant computational advantage over all 
other FIR LMS algorithms, for large values of n. 

Clipping the data in Eq, (2.47) gives 

n-1 

w(m+l) = wCm) + 2p, I 2 (mn+i) Sgn X(mn+i) (2.50) 

i=0 

Equation (2.45) is evaluated in the frequency domain as 
given by Eq, (2,48a). After evaluating Eq. (2.46), the 
computation of Eq, (2.50) is carried out in the sample domain 
itself . Though it seems that n sign checks would be required 
to compute Eq, (2,50), it can be implemented with only n sign 
checks. Therefore, evaluation of Eq, (2,45) by use of FFT 
algorithm, computing Eq, (2,46) and updating the weights in 
accordance with Eq, (2,50) is what constitutes a fast algo- 
rithm, herein called the fast clipped-^ata LMS (FCLMS) 
algorithm. 

If, on the other hand, the output is clipped in the 
weight update equation, Eq, (2.47) becomes 

n-1 

wCm+l) = W(m) + 2^ E [Sgn z(mn+i)] X{mn+1) . (2.51) 

*■ " i=0 

This equation obviously requires n sign checks to update 
the weights. Equations (2,46), (2,48a) and (2,51) result in, 
what may be called, the fast HYB algorithm. 
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It is also possible to clip the data in Bjs. (2.49c) 
and (2,49d) so that considerable saving in computation 
results, though at the cost of speed of convergence. Hie 
resulting algorithm may be called the clipped-<data HR LMS 
(CIIiMS) algorithm. If, instead, the output is clipped in 
these relations, what may be called, the clipped-error HR 
LMS algorithm results. If, in a similar way, the reference 
data are clipped in Eqs. (2.43a), (2,43b) and (2.43c), 
substantial saving in ccanpatation results, “Ihe consequent 
decrease in the speed of adaptation, in this case, may be 
compensated to some extent by increasing the value of p as 
there is no constraint on it for convergence. 

The algorithms proposed here have been listed in 
Appendix B, These were implemented on DEC— 1090 syston and 
tested for Gaussian inputs - white and coloured - and uni- 
formly distributed inputs. They need be tested for other 
non«^aussian inputs. 

It is now examined how the algorithms considered so 
far compare with each other in regard to speed of computation. 
As pjointed out earlier, the computational ccxnplexity of an 
LMS algorithm is primarily decided by the number of imalti- 
plications involved. Accordingly, this has been chosen as 
the basis of cxjmparison. Also, a block of n iterations will 
be considered instead of one. Furthermore, it will be assumed 
that the computation time requirements for "house-keeping” 
op>erations in the FFT algorithm is negligible in relation to 
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that required for rnultiplications, and that each complex 
multiplication requires 4 real multiplications. 

For real data, a 2n point EFT can be realized with 
an n point FFT and n complex multiplications [2-11] , Each 
n point FFT requires ^ log “ complex multiplications. For the 
evaluation of Eq. (2.48a), for example, a total of 
3(^ log ^ + n) + n complex multiplications will be required. 
Table 2,2 gives the relative computational requirements of 
various algorithms. The FIR LMS algorithms are compared with 
the LMS algorithm, whereas the CILMS algorithm is compared 
with the ILMS algorithm. It is seen that the CLMS and CILMS 
algorithms prove computationally efficient for small values 
of n, and the FCLMS and FCILMS algorithms for large values 
of n. 

The algorithms discussed thus far in this section 
are by no means exhaustive. Over the years several algorithms 
have been proposed which, though, compute the Wiener solution 
in an iterative' fashion, differ in some respect or the other.- 
Noteworthy among them are the higher-order algorithms [2-12] 
which have weight vector convergence curves different fratv 
those of the LMS algorithm. However, these algorithms do 
not speed up convergence in an absolute sense. White [2-13] 
has reported a generalized HR LMS algorithm of which 
Feintuch’s algorithm discussed previously is a particular 
case. A comparative discussion about some of the earlier 
algorithms may be found in [2-3] , 
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Table 2,2 : Comparison of Computation. 


n 

FLMS 

CLMS 

FCLMS 

CILMS 


(10 log ~ + 28) X n 

2- 

n 

(6 log ^ + 16) X n 

2n^ 


2n^'^ 

- - 


2n^ 

16 

1,8125 

0.5 

1.0625 

0.5 

32 

1.0625 

11 

0.6250 

tt 

64 

0,6094 

tl 

0.3593 

If 

128 

0.3438 

tl 

0.2031 

tl 

256 

0.1914 

11 

0,1132 

tf 

512 

0.1055 

it 

0,0625 

it 

1024 

0,0576 

If 

0.0341 

II 
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SECTION 3 

ADVANTACSS AD© LIMITATIONS 

As was pointed out in Section 1, the main advantage 
of ANC ' s lies in their ability to learn the signal character- 
istics and track them if they vary slower than the adaptation 
rate of the system. In circumstances where the noise cance- 
lling technique is applicable, levels of noise rejection are 
often attainable that would be difficult or impossible to 
achieve by direct filtering. Examples include adaptive 
notch filter, self-tuning filter and bias weight filter. 

The advantages c£ ANC's are illustrated in Section 4. 

Besides the requirement for a suitable reference 
input, the factors that limit the system performance are; 
uncorrelated noise ccraponents present in the inputs, signal 
components present in the reference input, finite length of 
the ad^tive filter, misad justment due to gradient esti- 
mation noise and errors arising from digital implementatiCTi. 

The misad justment was discussed in secticn 2, The 
other performance limitations are described in this secticn. 
The discussion here closely follows [2-1], For stationary 
stochastic inputs, the steady-state perfoimance of adaptive 
filters closely approximates that of fixed Wiener filters, 
as was seen in section 2. Thus the analysis of the short- 
comings of ANC's is facilitated by the Wiener filter theory. 
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The Wiener filter is assumed to be unc cns trained ; 
i.e.v it is conposed of an infinitely long^ two-sided (non^ 
causal), tapped delay line. The optimal impulse response 
is obtained frcm the discrete Wiener-Hq?f equation [see 
Ecj. (2.8)] ; 

CO 

I W*(i) R^(l - i) = 

X= — .CO 

where 

+ D] 

and 


Rdx(l) - E[d(j) x(j + D], 


The z transform oE Eq, (3.1) yields the transfer function: 


w*(z) 




(3.2) 


where S^( 2 ) and power spectral densities, are 

the z transforms of ^^( 2 ) and R^^(z), respectively. The 
^plication cf Wiener filter theory to noise cancelling is 
now considered. 


3,1 Effect oE Uncorrelated Noise Cartpaients in the Inputs 

Figure 3,1 shows an adaptive noise canceller, 
which has, besides the usual inputs, a pair of noise 
comp men ts, m^(j) and that are uncorrelated with 





, J 



Fig. 3.1: ANC with uncorrclated noise components 
in the inputs • 



Fig . 3 . 2 1 Equivolent channel representation . 
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each other, with s, with n(j), and with h(j) * n{j). The 

uncorrelated noise conponents and the signal are assumed to 

have a finite power spectrum at all frequencies. For the 

purpose of analysis all noise propagation paths, represented 

by H^iz) and H^Cz), are assumed to be equivalent to linear, 

time-invariant filters. The transfer functioi c£ the signal 

propagation path is represented by H^(z). 

The pictorial model of the channel in Figure 3.1 

may equivalently be represented by the model in Figure 3.2; 

H (z) 

where H(z) = optimal, unconstrained transfer 

function of the adaptive filter, assuming that the adaptive 
process has converged and the minimum mse solution has been 
found, can be obtained from Eq. (3,2) as follows: The 
inputs to the ANC, from Figure 3,2, are 

d ( j ) = s ( j ) + n ( j ) + m^ ( j ) 

and 

x(j) = h(j) * n(j) + m^(j) , 

Denoting the power spectral density of the noise 
S (p) and that of n(j) by s^(z), the filter's input spec- 
trum, S„ ( z ) , can be written as 

S^(z) = S (z) + S (z) |h(z) ] , 

X m n 

!!/v 

The cross-spectral density, given by 


(3.3) 



Ccmbining Eqs. (3,2), (3,3) and (3,4), we obtain the Wiener 
transfer function 


( 2 ) = 


Sj^(2) H(2“ ) 


X 


(3,5) 


Let the uncorrelated-to^correlated spectral density 
ratios of the inputs be defined as 


S„ (z ) 


A(2) = 


m 


d 


S„(2) 


(3.6) 


and 


B (z ) = 


S (z) 


1 H(z) 


(3.7) 


Ccmbining Eqs. (3.5) and (3.7) yields 


W*(z) = 


H(z) b(z')7 


(3.8) 


The performance of the noise canceller may be 
evaluated in terns cf signal-to-noise density ratios, 
defined as 


p^(z) 


Output signal spectral density 
Output noise spectral density 


and 


P^(z) 


Primary signal spectral density 
Primary noise spectral density 



Therefore, 


P 

z 


(z) 



S_(2) + (z) 

n 

— 

z 


or 


+ A(z)] 


®n 


where iz) is the output noise spectral density corres^^ 
z 

ponding to the output noise, n (j), given by 

z 


n^Cj) = m^(j) .. w*(j) * m (j) + [1 - h(j) * w*(j)] 


n 


The po\<\rer spectral density c£ the output noise is 


^n + I w^(z) f (z) 


X 


+ I [l - H(z) w*{z)] I s (z) . 


Conbining Eqs, (3.6), (3.8) and (3.10) yields 


S„^(z) 


S (z) A(z) + S (z) — r 

n n 1 + B(z) 


From Eqs. (3.9) and (3,11), we obtain 


P^(z) 

7^ 


fi + A(z)i r 1 + 3 (z) 1 ■ 

A(z) + A(zT B(z) + B(z) ' 


3.9) 


(j) . 


3.10) 


3.11 ) 


The above expression provides a general descrip- 
tion of ideal noise canceller performance, and as such 
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enables one to estimate the level of noise areduction possible 
in an ideal noise cancelling system with inputs containing 
uncorrelated noise components. It is apparent from the 
equation that the level c£ noise reduction is limited by the 
uncorrelated=tO“Correlated noise density ratios at the 
primary and reference inputs. This is perhaps best illus- 
trated by the follov;ing specific cases: 


Case 1 




When A(z) = 0^ 
1 


= 1 + 


B(z) * 


Case 2 


P^Cz) 

■5^ 


Waen B(z) = O, 


= 1 + 


aTzT 


Case 3 


I€ien A(z) = B(z) = O, 


These cases demonstrate the desirability of low 
levels of uncorrelated noise compcnents in the inputs. The 
smaller the values oE A(z) and B(z), the larger is the 
ratio P (z)/pj(z) and, hence, the more effective is the 

Z Cl 

action of noise canceller. It may be noted that when 
m (j) = 0, Eq, (3.5) reduces to 


w*(z) = 


hTTT 
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V7hich. shows that the adaptive filter essentially adapts to 
the inverse of the channel characteristics. 

^ Effect of Signal Compcaients in the Reference Input 

In certain situations the available reference 
input to an ANC may contain low-level signal ccmpcnents, 
besides the usual correlated and uncorrelated noise comp-- 
onehts . Ihese signal components will, no doubt, cause seme 
cancellation of the primary input, thereby distorting it, 

A quantitative analysis of such a distortion is considered 
here. For simplicity, it is assumed that the uncorrelated 
noise ccmpoients in the inputs are absent. It is also 
assumed that the signal propagation path between the source 
and the reference input (see Figure 3,2) is characterized 
by a linear, time-invariant filter, G(z), 

The reference power spectral density, S (z), in 
this case, is given by 

S_(z) = \g{z)\'^ S^(z) 1h(z)1^ S„(z) 

IX. S ii 

and the cross- spectral density, by 

S^^(z) = G(z”^) Sg(z) 4- H(z“^) S^(z). 

'Ihe Wiener transfer function is, therefore, 

G(z"^) S (z) + H(z"^) S_(2) 

W*(z) = ^ ^ — *2 * 

1 G(z) r S_(z) + 1 H(z) 1 S (z) 

S 11 


. 12 ) 
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Tile spectral density o£ the signal canponent in 
the output is 

1 1 - G(z) W*(z) 1 S (z) 

s 

and that of the noise ccmpcnent in the output 

ll “H(z)W*(z)l S (z) . 

n 

Ihe output signal=to-noise density ratio is, therefore^ 

|l - G(z) W*(z) 1 s^(z) 

^z^^^ ” |1 - H(z ) (z)| Sj^(z ) * 

Substituting for W*(z) and cancelling the canman terms yield 

.2 

S„ 

(3.13) 


1 H ( z ) 1 S„ ( z ) 

oAz) = L-a — 


G(z) j Sg(2) 


The spectral density of the signal canponent in 
the reference input is 


G(z) 1 Sg(2) 


and that of the noise canponent in the reference input 


tH(z) 1 S^( 2 ) . 

The signal-to-noise density ratio, p^(z), at the reference 
input is, therefore, 

1g(z) j ^ S„(2) 

P^(z) = 


H(z)| 


(3.14) 
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Fran Eqs. (3.13) and (3,14), we obtain 


P (z) 
2 


■ 


(3.15) 


Ihe above relation shcuvs that, assuming the Wiener 
solution vector to be unconstrained and the noises in the 
two inputs to be mutually correlated, the signal“to*-noise 
density ratio at the noise canceller output is the reci- 
procal o£ the signal- to-n Oise density ratio at the reference 
input. If P„(z) = O, then p_(z) = “. Thus, the lower the 
signal conponents in the reference input, the better is the 
performance c£ the noise canceller. 

The output signal distortion caused by the signal 
component propagated through the adaptive filter is measured 
in terms csf a dimensionless quantity, D(z), which is defined 
as the ratio of the spectral density of the output signal 
component propagated through the adaptive filter to the 
spectral density of the signal component at the primary 
input; i.e, 

D(z) = 1 G(z) V?*(z) - (3.16) 

If 1 g( 2)| is small, it may be reasonable to assume that 

2 

1g(z) { i Then, from Eq. (3.12), 

1 G(2) 

> HUT ' 


D(2 ) 
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or, alternatively. 


D(z) 





(z) 

nj ' 


Hie above relation means that, under the stated 
assumptions, low signal distortion results from a high 
signal“to«=noise density ratio at the primary input and a 
low signal- to=noise density ratio at the reference input, 

Ihis shows that signal conpcnents of lov; signal-to-noise 
ratio at the reference input do not render the application 
of adaptive noise cancelling useless. If the reference 
input contained cnly signal conpcnents but no noise compo- 
nents, correlated or uncorrelated, then the signal ccmponents 
would ccmpletely be cancelled. However, if the reference 
input is properly derived, this ccnditicn will not occur. 


3.3 Finite Length, Causal Approximation 

The Wiener solution considered so far in this 
section is based on unconstrained Wiener filter theory? i.e., 
the filter calls for an infinitely Icng, two-sided tapped 
delay line, which is, of course, not feasible to realize. 
Since impulse responses of practical systems tend expcnen- 
tially to zero with time, approximate realizations by 
means of finite length tapped-delay lines do became feasible. 

In circumstances where a noncausal response is 
desired, a suitable delay may be provided in the primary 
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input so as to cause the peak of the impulse response to be 
located at the centre of the delay line, it has been pointed 
out [2"l] that a delay equal to about half the time delay of 
the adaptive filter produces the minimum output noise power, 

3 . 4 Errors Arising from Digital Implementati on 

In an implementation based cn digital hardware, 
errors due to truncation or roundoff are inevitable. When 
analog signals are processed, the quantization becomes yet 
another source of error. If these errors are not kept within 
bounds, the adaptive system might become unstable. It has 
been shown [3~1 ] that if the adaptation constant and the 
upper bounds on errors arising from quantization and roundoff 
are sufficiently small, then the convergence of the weight 
vector is ensured. An exact expression for the condition 
may be found in [3-1], 

To summarize, the analysis presented in this 
section emphasises the requirement f or a suitable reference 
input. In general, some ingenuity is required in deriving 
the reference input such that the undesirable components are 
kept at a low level. Also, it is possible to realize a 
non “-causal response by providing an appropriate delay in 
the primary input. Finally, the parameter ^ has to be so 
chosen as to, besides ensuring convergence, keep the errors 
arising from digital imp 1 amen tat ioi and misaS justment 
sufficiently low an the one hand and the adaptive process 
sufficiently fast on the other. 
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SECTION 4 

APPLICATIONS AND SIMULATION RESULTS 


Adaptive noise cancelling has been used in speech 
signal processing, pattern recognition, adaptive antennas, 
and medical electronics; the technique can also be used in 

the removal of power line interferences, tape hum, and in 

« 

the detection of very*- low-level signals corrupted by broad- 
band noise [ 2-1, 4-1]-. In this section, several typical 
applications of ANC's are discussed and simulation results 
presented, which show the wide range of applicability of 
noise cancellers, Ihe LMS algorithm was used in the simi- 
lations, which were carried out on the DEC-1090 system, using 
Fortran-10 language. For simplicity, the uncorrelated noise 
components in the inputs and the signal components in the 
reference input are assumed absent, though their presence 
would alter the results presented here only to the extent 
discussed in Section 3, 

4,1 The ANC as a Notch Filter 

The Conventional method of eliminating a single 
frequency sinusoidal interference from a signal is through 
the use of a notch filter. The ANC could also be employed 
to remove an unwanted component from an incoming signal. 

The advantages of such an adaptive notch filter are: easy 
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control of bandwidth, an infinite null, and the ability to 
track the exact frequency of interference. 

The primary signal can be of any kind - determin- 
istic or stochastic. Ihe noise in the primary input is 
assumed a cos(w^t + ©); here analog variables are used for 
notational convenience, ihe reference ccmponent is denoted 
by b cos(o^t + ^2'). Two degrees of freedom or are 

required for the filter to achieve the necessary gain and 
phase for cancellation. Since deterministic signals are 
involved here, the problem of noise cancelling may be reform- 
ulated as one of minimizing the sum of squared-errors over 
time. Assuming that .the primary input is zero-mean ergodic, 
the optimum weight vector may be found by minimizing the 
sum of squared errors. 


Lt 
in 00 


1 

m 


m-1 

j=0 


e^(j) 


- itW“X 
xa-yo> j=0 




Here the signals have been assumed one-sided. 

Following the same procedure as in Section 2, the 
optimum weight vector may be obtained by solving the linear 
simultaneous equations 


Rv = 
X — 


Sdx 


(4.1) 


R 


X 


A 

~ m -*• ® 


m-1 

I 

3=0 


3 ^ 3 ) 


Where 


1 

m 


(4.2) 
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Sax 5. 5 I '3<j> x'j) 

m ^ j=0 

where the relations (4.2) and (4,3) are similar to the 
empirical relations that result from the assumption of 
ergodicity of the stochastic inpats. 

If it is assumed that the signal is uncorrelated 
with the noise, then Eq. (4.3) may be rewritten as 


m-1 


■Sdx 


-t 

m I . 

j=0 


in -V CO 


It may be noted that n^(j) and n^(j), where n^(j) = x(j), 
are deterministic and Eqs, (4.2) and (4,3) define the 
corresponding discrete-time correlation functions. For 
periodic signals with period m, these equations may be 
rewritten as 


and 


R 


X 


1 

m 


m-1 

I X(j)x"(j) 

j=0 


. m-1 

Sax = S • 

Since two taps will suffice in the case of single frequency 
sinusoid, the correlation functions reduce to 


R. 



2 

^ cos “ T 
2 o 



u T 
o 
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and 


■2dx “ 


cos(& - { 2 r) 

3b 

■5“ cos(<j3 T - & + 

Cm O 


where T is the delay between the taps. If the delay is 
chosen as T*/4^ where T' is the period of the signal, then 
the optimum weights are furnished by 


W 


cos(Q - gf) 
^ sin(©- - gf) 


(4.4) 


To illustrate the effectiveness of the single- 
frequency noise canceller, a simulation was carried out with 

d(j) = s(j) + l,5 sin (-| — j) 


oir 

x(j) = Sin (“~ — j + 1“ ) 

y = 0,0005, and n = 2, where 3 ( 3 ) is a coloured Gaussian 
sequence obtained by passing a white Gaussian sequence 
( 0 , 1 ) [i.e,, with zero mean and unit variance] through a 
single-pole low pass filter with the pole at 0.5. Ihe 
values of the theoretical as well as the simulated weights 
are tabulated in page 4—18 [ Table 4,1 ]. The siimalated 
weights are in close agreement with the theoretical ones. 
The plots of the signals taken at various nodes after the 
steady-^tate had been reached are shown in Figures 4.1-1 to 
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The numbers to the left of the plots represent the 
amplitudes of the corresponding sequences. Also the number 
of • + * signs against each number is proportional to the 
corresponding amplitude. 

Approximate expressions for the notch bandwidth and 
quality factor are given by [2-1] 

, „ t.2 

Notch bandwidth — T — 

and 

u T 

Quality factor 'v, — — ^ ^ 

" 2vb 

respectively. From these expressions, it is seen that by an 
appropriate choice of V , a narrow and infinite null can be 
achieved in the spectrum of the primary input. It may be 
noted that in case the delay is not chosen as T*/4, the 
weights will converge to values different from those given 
by Eq. (4.4). 

In a second simulation, 512 discrete sinusoids were 
added to fom the primary injut, and the reference input was 
one of the primary components, but with a different phase 
and amplitude, Ihe normalized power spectrum at the output 
after the weights had converged is shown in Figure 4.1-7. 

The plot shows an excellent rejection of the required 
signal component. When the reference signal frequency was 
slightly varied, even then there was cancellation, but the 
weights converged to a dynamic rather than a static solution 
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’’tumbling" at the difference frequency; in other words^ the 
ANC behaves like a modulator converting the reference 
frequency into primary frequency. 

In a third simulation, an uncorrelated Gaussian noise 
sequence was used as the primary input. Ihe reference input 
and the processing parameters were the same as in the first 
simulation. An ensemble average of 200 power spectra at the 
noise caiceller output is shown in Figure 4,1-8. Ihe 
resolution of the EFT used to obtain the power spectra was 
512, what ace known as in the jargon, spectral bins; i.e., 
the FFT was of 512 points. It was observed during the 
simulation that a fast adaptation leads to cancellation of 
adjacent frequency components. However, if it is kept slow, 
the effect is generally insignificant. 

The results presented so far demonstrate the 
effectiveness of single frequency noise cancellers. In a 
similar way, a multiple frequency noise canceller can also 
be constructed. The second simulation was modified to 
remove three more components from the primary input. 

Figure 4.1—9 shov#s the corresponding power spectrum. 
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^•2 The ANC as a Highpass Filter 

It is also possible to rmove from the primary input 
any slow drift or bias component along with the noise compo“ 
nents. This is accomplished by including one more filter 
weight, known as bias weight, and setting its input to a 
Constant value that will represent a component in the 
reference input correlated with the bias or drift in the 
primary input, so that the latter is removed along with other 
correlated components. This is illustrated in the present 
example , 

A triangular wave s ( j ) was added to a colouce|3 
Gaussian sequence to form the primary input 

d(j) = s(j) + 1.4 n(j) +2 

where the constant 2 is the bias. For simplicity, the 
reference input was obtained by passing the coloured Gaussian 
sequence through a third order HR (infinite impulse response) 
filter representing the channel with 

n^(j) = 0.7 n^(j-l) + 0.2 n^(j-2) + 0.05 n^(j“3) 

X X X X 

+ 0,7 n( j) 

where n (j) is the channel output^ As the channel charac- 

X 

teris tics are known, it can be shlowi that the theoretical 
optimum weight vector given by 
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The ANC with stationary Inputs 

The purpose o£ this treatment is to illustrate the 
various design rules given in Section 2. Two independent 
zero-mean white Gaussian sequences, qCj) and n( 3 ), of powers 
0.01 and 2,0, respectively, were combined to form the signals 

s(j) = 0.99 s(j-l) + q(j) 

and 

n„(j) = 0.6 n^(j-l) +0.2 n^(j-2) - 0.3 n^(j-3) 

-r> ^ .X X 

- 0,1 n^(j“4) + 0.7 n^(j) 

where 

n^(j) = 0.1 n^(j-l) + n(j) , 

"nie inputs to the noise canceller were 
d(j) = sCj) + 1.4 n^{ j) 

and 

x(j) = n^(j) . 

Using five weights and a value of 0,001 for |j, , 
a simulation was carried out the plots of vhich are shown 
in Figures 4,4-1 to 4,4-6, Figure 4,4-7 shows an individual 
learning curve obtained from Eq, (2,7) corresponding to the 
current weight vector that resulted from adaptation. 

Figure 4,4-8 shows a scmewhat smooth learning curve obtained 
by averaging over an ensemble of 120 simulations, each 
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starting with the same initial vector W(0) = o and each 
having a different input derived from the same "statistical 
population*" The interval between successive ordinates in 
both the plots is lo iterations, Ihe theoretical results 
and the results obtained from simulation are presented and 
compared in what follows, 

The estimates of the correlation matrices are: 

4.2769 2,8866 1.5657 -0.0529 -1.0076 

2.8867 4,2448 2.8653 1.5602 -0.0617 

1.5658 2.8653 4.2968 2,9156 1.6076 

-0.0529 1.5602 2,9156 4.2752 2.8839 

-1.0076 -0.0617 1.6076 2.8834 4.2615 ^ 

and 

4.2308 
0.4578 
0.0461 
-0.0025 
{_ -0.0015 

The theoretical weights obtained from Eq, (2.8) and the 

simulated weights measured at different instants are given 

in Table 4.3. The eigenvalues of are given by 

10.9182 
7 .2007 
0.7758 
1,5683 
0,8920 
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Ihe ANC produced about 2,5% more rase th^ the 

■< 

Wiener filter; but then, the Wiener filter could have been 
designed only with the complete, knowledge of t±te second - 
order statistics of the signal and noise. 
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4*5 Tfife j^C with NonstationarY Inputs 

Soiae of the best applicatibhs of ANC's will doubtless 
be made tx) tte filtering of nonstaticmary inputs, Vftxen a 
slowly-varying process is applied to an ANC, the latter 
tra3ks tlte characteristics of the f ojmer in a sub-optimura 
way f 2-2 3* Means of describing the behaviour of ad^tive 
syBixms with ncmstationary Inputs is currently a subject of 
intensive research. 

Two siimili^ions'were carried out to dentonstrate the 
effectiveness of ANC's with nonstationary injuts. The same 
signals and processing parameters/ used in the previous 
simulation, were used in the present ones, iHit for the 
differences as stated in what follows. 

In the first simulation, the varismce of n( j) was 
modulated fcy a square-wave with period T Once the 

initial transients were over, the adaptive filter was seen 
to track the signal characteristics with little change. 

In the second simulation, the variance of q(j) Was 
stepped up by 20 dB and 30 dB after the ste^y -state had 
been reached. We shall define' the convergence time as the 
time required for the filter weights to settle within 5% of 
their steady-state values, Ihe convergence times of the 
weights for different ji’s and signal power levels are 
tabulated in page 4-44 ('Table 4,'4), 

Next, the channel coefficients were slowly varigd. 

Ihe plots of one of the channel coefficients and the 
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corresponding filter weight that tracks it are given in 
Figures 4,5-1 and 4,5-2, respectively. It is seen that the* 
weight tracks the channel variations in a suh—optiinuni way. 
Although for simplicity the zeros of the transfer 
function of the channel models considered in this section 
have been assumed absent, their introduction will increase 
the number of filter weight considerably. A good approxima- 
tion of a sibgie pole filter by an FIR filter, for instance, 
requifes as many as 10 filter weights [2-9] j on the other 
hand, an IIR filter will require just one weight. 


'Table 4,4 ; Convergence time for nonstationary inputs. 


„1. - J 

f 

Signal power change I 

I 

t 

v - - - - - 

{A 
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1 Settling tame 

I (No, of iterations) 
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t .. . 


0,005 

243 

20 dB 

0,001 

297 


0,00C6 

356 


0.001 

365 

30 dB 

0.0004 

407 


0.0007 

458 
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SECTION 5 


MICROPROCESSOR I]>$>LEMENTATION 

Having discussed the applications of ANC's in the 
previous section, we new examine the hardware irr^lementation 
o£ real-time noise cancellers by means of microprocessors. 

5,1 Choice o£ Hardwaire 

The basic functional block of an ANC is a program- 
mable transversal filter whose tap weights are modified 
according to a pre-assigned algorithm. Basically, there are 
three distinct realization techniques for a prograiranable 
filter: all digital approach, all analog approach and hybrid 
epproach. The first approach has the limitation of compu- 
tation time, which limits the signal bandwidth that can be 
handled by the processor, and the requirement of A/D arK3 D/A 
converters to process analog signals. The seccxid approach 
has the problems of offset, gain, and other performance 
limitations. The third approach is a corrpromise between 
these two ^proaches in that it uses analog irput signals 
and digital tap weights. Here, the weighting of the refer- 
ence signals is accomplished by multiplying-D/A ccaiverters. 
Ihe programmable transversal filters were realized in the 
past by means of hard-wired logic circuits analog 
components . Recently, single chip CCD adaptive filters [2-5] 
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Table 5,1 : Salient features of microprocessors. 


Processor 

INTEL 

8080A 

INTEL 

8086 

TI 

9900 

ZILOG 

8000 

ISI- 

11 

Technology 

NMOS 

NMOS 

NMOS 

NMOS 

NMOS 

Word size 
data/instruction 

8/8 

16 /l6 

16/16 

16A6 

16/16 

Number of 
instructions 

78 

97 

69 

110 

lOO 

Maximum clock 
f requency /phase 

2.6/2 

5A 

4/4 

8/1 


Instruction time 
sh or te s t/1 cn ge s t 

1.5/3.75 

0.4/37 

2/31 

0.75/90 


TTL compatible 

YES 

YES 

YES 

YES 

YES 

BCD arithmetic 

YES 

YES 

NO 

YES 

NO 

On-chip intermapt 
levels 

1 

1 

16 

1 

1 

Number of general 
purpose registers 

8 

8 

16 

16 

8 

DMA capability 

YES 

YES- 

YES 

YES 

YES 

Prototype system 
availability 

YES 

YES 

NO 

YES 

YES 

Package size (pins) 

40 

40 

64 

40/48 

SBC 

Voltage required 
(volts) 

5, 12 
-5 

5 

5^ 12 
-5 

5 

5 

Assembly language 
development system 

YES 

YES 

YES 

YES 

YES 
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The zilog 8000 excels the other processors in respect of 
instructim cycle time, instruction set and number o£ 
registers. For this project, an Intel 8080 processor was 
chosen because of its ready availability and the standard 
support software available with it, 

Iniplementation Details 

Wc now need to examine which algorithm to choose 
for implementation. Among the algorithms discuss^ in 
section 2, the LMS algorithm and the CLMS algorithm are 
amenable for microprocessor implementation, Ihe latter 
algorithm involves less "number crunching" on the processor, 
though it converges slower than the former by a factor oE 
w/2. In order to facilitate signal processing in real-time, 
the ccmputaticnal burden on the processor must be minimized, 
so, the 01)143 algorithm was chosen for implementation, 

Ihe flowchart for the CLMS algorithm is shown in 
Figure 5,1, An assembly lan^age program was developed and 
tested on ECIL's microcomputer, MICRO-78, Ihe computer has 
been built around an Intel 8C80 processor, and has, among 
.other facilities, a resident assembler and an on-line 
debugging system (ODs). Description of the MICRO-78 and 
the Intel 8080 processor may be found in [5-4], 

To minimize the errors arising from the digital 
implementation, multiple-precision arithmetic has been used, 
Ihe I/O data are represented by 9 bits (this will correspond 




Rf.5.i: Rowctwl for the clipped-cJata LMS olgorithrn. 
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to a coarse quantizaticn when analog signals are processed), 

the weights by 16 bits and the error by 24 bits. If the 

constant p' , where t' = 2y , is chosen as a paver of 2, 

then multiplication by y' is tantamount to a series of 

shift cperations. Accordingly, y' is chosen as 2“^^, and 

«20 -“I 

it can be decreased down to 2 in steps of 2 , Ihe 

number of taps can be preset. Ihe specifications of the 

3080“processor“based Airc is given in Table 5.2 (page 5-9). 

A listing of the assembly language program is 

given in Appendix A. The following sequence of cperations 

are performed by the program: To begin with the tap weights 

W and the noise samples x are initialized to zero; 

the input data are then accepted, and the convoluticn 

suramatim I w(i) x(i) is formed; the sum is subtracted 
i 

fron the primary input to yield the output z; next, a 
series of right shift cperations are perforraod on a, and 
the shifted value is added or subtracted fron w(i) depending 
cn sgn x(i); after updating the weights, the next input 
data are accepted for the subsequent iteraticn. By suitably 
manipulating the addresses, the program avoids shifting cf 
reference data samples corre spending to their prepagation 
through the transversal filter. 

5,3 Hardware Scheme 

With a minor change in the program, the ANC can 
be inpleraented as shown in Figure 5,2. Ihe progran may be 



RAM I I AOL I— *1 TSO 



Fig. 5.2 : Hardware configuration. 
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stored in EPROM's and the weight values and the reference 
samples in RAM's, A programmable peripheral interface (PP I) 
serves as the interface between the processor and the A/D 
converter . 

For addressing thePPI's in order to read the 
inputs, memory mapped I/O scheme may be ertployed, thereto 
treating I/O devices as memory locations. In 8080, the 
instruction MOV A,M would transfer one byte of data from 
a PPI to the accumulator, if its address is stored in the 
register-pair, HL. Data output could be accomplished by 
means of an address decoder logic (ADD) and a tri-state 
gate (TSG). 

With the help of a logic state analyzer (Type: 

HP 1610) timing measurements were made on the developed 
software routine. The camputation times per iteration for 
different numbers of taps are given in Table 5,3, Ihe 
program segment MULTI takes a raaximiM of 700 jis. Ihe clock 
period of the processor was 0.595 p,s (sftinijnum clock period = 
0.385 jis ) , 

Considerable speed improvement could be achieved 
by replacing the software multiplier by a hardware parallel 
multiplier. Also, multiplication by logarithmic table 
look-up [5-6] and combinatorial methods [5-7] show promise 
here. A potentially feist inplementation may use prog rartm- 
able CCD's and hard -wired logic circuits. Ihe microprocessor 
based ANC is, of course, suited to only low-frequency signal 
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T^le 5,1/ 


ANC specif icaticns, 


I/O word length 

• 

# 

9 bits 

AdeptatiMi constant ' 

* 

♦ 

3-14 _ 3-20 

Number of taps (min.) 


1 

Iteration period (min.) 

«r 

4 ms 

sample rate (max.) 

# 

250 H2 

Weight vector update rate (max.) 

m 

m 

250 Hz 

Arithmetic 

• 

Fixed point 


Table 5,2 : ANC conputation time. 


T 


No, oE 
taps 

t 

1 

1 

1 

1 

1 

f ... 

Computation time/iteration 
(ms) 

2 


3.790 

5 


7.298 

16 


21.790 


32 


48.820 
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extraction. With a slight modif icatim in the program, 
16-bit data can be processed. However# it may be preferable 
to employ 16-bit processors . 
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SECTION 6 

CONCLUSIONS 


6.1 Summary 

The caicept o£ adaptation in noise cancelling has 
proven to be a powerful and versatile means of signal 
proressing where precise a priori filter design is imprac“ 
tical. As was seen, no a priori knowledge of signal and 
noise characteristics is required; pertinent knowledge of 
their characteristics is acquired through a process of 
adaptation. Implementation of noise cancellers in their 
most general form [6-1] is, of course, beset with severe 
practical constraints in that they are ccmputaticnally 
intractable; however, with the approximation introauced by 
the LMS algorithm, their realization turns out to be consi- 
derably simplified. 

For the LMS algorithm, simplified design equations, 
which are at present available in a scattered form in the 
literature, have been presented in a unified manner. Towards 
this unification, the main features of the treatment presented 
here are the following: 

1) Derivaticn., from first principles, of the 
expression for the time constants of the weight vector. 

2) Derivation of the expressions for rate of 
adaptation and condition for convergence for the 



n=^imensiaial case of the GLMS algorittai. 

3) Relative evaluaticn of the ccmputaticns involved 
in the various adaptatim algorithms ccnsidered in this work. 

4) A new method for reduction a£ computation and 
its applications to some of the existing algorithms. 

5) P roviding quantitative arguments for seme of 

the explanations given in the literature on qualitative lines. 

6) Examining the possibility of implementing real- 
time noise cancellers by means of microprocessors. 

Besides the LMS algorithm, some of the recent algorithms 
were also discuss©^ and compared in the work presented here. 

In addition to theoretical analysis/ simulation 
studies play an important role in systems such as ANC’s. 
Extensive simulations were carried out bearing on different 
aspects/ some of them duplicating previous work and some 
designed specifically in the course of this work. On the 
whole, the results of simulation presented here should 
serve to provide a clear understanding of the basic principles 
and also reveal some of the finer features of ANC's. 

A quantitative analysis on the advantages airf 
limitations of noise cancellers was included to indicate 
the level cf performance attainable in practical environ- 
ments- As for their realizatioi, the microprocessor based 
ANC ccxistitutes a practical possibility for low frequency 
applications. Moreover, with a digital processor to 
implement the ANC's, it may also be possible to program 
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the design rules for the optimal choice of paraneters; in 
that case the ANC’s may truly be said to be "self -designing" 

6,2 scope for Fufrther Work 

The cpplicatiaas of ANC’s to areas wherein fixed 
filters are now emplc^ed may be systematically explored, as 
this would ease the burden of stringent prior design on the 
part a£ the designer. Besides, an attempt may be made to 
study and compare the various properties of the numerous 
adaptive algorithms available in the literature. Also, the 
MLMS algorithm, which is otherwise superior to the other 
algorithms, needs further attention in respect of its simpli 
fied realization. Finally, it may be mentioned that, as 
pointed out in [2-2], there is considerable scope f or 
theoretical research in what has been called "the statis- 
tical theory a£ adaptation". 



